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(57) Abstract: A system and method or binding MPLS labels to virtually concatenated S0NE17SDI I transport connections is pro- 
vided. A system and method or binding MPI^ labels lo nxVI' connections is provided and includes (1) classirying packets; and (2) 
mapping packets with a same rorurard equialencc class to a same nxVT connection. IN a further embodiment, the mapping includes: 
(a) associating a certain bandwidth of a connection between an external router and an add-drop muliiplcxcr beiwecn a source node 
and a destination node: and (b) creating an nxVT connection between the source node and ihc dcsiination node, thereby creating a 
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labels 10 nx VT connections is also provided and includes mapping packets with the same MPLS label lo the same nx VT connection. 



atSOODOc <WO ^0217S42A2J.> 



wo 02/1 7542 PCT/US01/2653J 



5 

SYSTEM AND METHOD OF BINDING MPLS LABELS TO 
VIHTUAI.Jt.Y CO^qCATgWATPD SONET/gD^ TRANSPORT 

CONNECTIONS 

10 



15 



SPECIFICATION 
RELATED APPLICATIONS 

20 This application is related to U.S. Provisional Application No. 

60/228,008, filed on August 23, 2000, to U.S. Provisional Application No. 
60/272,793 , filed on March 1, 2001, to co-pending and commonly assigned 
U.S. Patent AppUcation No. (Number to be assigned) with Attorney Docket 
Number 55369-014, filed on August 23, 2001, and to co-pending and 

25 commonly assigned U.S. Patent AppUcation No. (Number to be assigned) with 
Attorney Docket Number 55369-015, filed on August 23, 2001 . The contents 
of U.S. Provisional AppUcation No. 60/228,008, filed on August 23, 2000, of 
U.S. Provisional Application No. 60/272,793, filed on March 1, 2001, of co- 
pending and commonly assigned U.S. Patent Application No. (Number to be 

30 assigned) with Attorney Docket Number 55369-014, filed on August 23, 2001, 
and of co-pending and commonly assigned U.S. Patent Application No. 
(Number to be assigned) with Attorney Docket Number 55369-015, filed on 
August 23, 2001, are hereby incorporated by reference. Tliis application 
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claims priority to U.S. Provisional Application No. 60/228,008, filed on 
August 23, 2000. and to U.S. Provisional Application No. 60/272,793, filed on 
March 1,2001. 

FIELD OF THE INVENTION 
5 The present invention relates to optical networking. More particularly, 

the invention relates to a system and method of binding MPLS labels to 
virtually concatenated SONET/SDH transport connections. 



BACKGROUND OF THE INVENTION 

10 Both SONET (See Synchronous Optical Network (SONET) Transport 

Systems: Common Generic Criteria. GR-253-CORE, Issue 2, Revision 1 . 
December, 1997.) and SDH (See Memational Telecommunication Union. 
Network Node Interface for the Synchronous Digital Hierarchy. 
Recommendation G.707. March, 1996.) enable the use of virtual 

1 S concatenation to support both the dynamic resizing of transport trunks and the 
grooming of traffic. More recently, advances in the transport of routed 
datagram traffic leveraging the research and experience of ATM has resulted 
in the standardization of MPLS (See Internet Engineering Task Force, 
Multiprotocol Label Switching Architecture. IETF Draft Document August, 

20 1999 and http://www.ietf org/intemet-drafts/draft-ietf-mpls-arch-06,txt.). 
This work allows network devices to employ a standards-based method by 
which packet trafBc can traverse a network, while receiving a previously 
agreed upon QuaUty of Service. 

Additionally, there is cunrently an effort underway by IETF (See 

25 Generalized MPLS - Signaling Functional Description; 

ftp;//i^.ietf.org/intemet-drafts/draft-ietf-mpls-generalized-signaling-05.txt), 
which attempts to adapt the control plane semantics of generalized MPLS to 
allow vendor-indqjendent provisioning of services at the transport layer. 
Traditionally, design q>proaches for large-scaled packet networks 

30 embrace the use of multiple control planes. A typical example would be 
mapping an IP overlay on top of a Layer-2 ATM network. Such networks 
typically suffer a discontinuity between the higher layer connectionless IP 
control plane running an IP routing protocol (whether it be an interior gateway 
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protocol, such as OSPF, or an exterior gateway protocol, such as BGP), and 
the lower layer connection-oriented ATM control plane running its own 
routing protocol (PNNI). Numerous approaches have bem submitted to 
standards bodies to address this issue, each with its various merits and 
5 demerits. In addition to the discontinuity between the Layer 2 and Layer 3 
control planes, the underlying transport layer connections, using such 
technologies as SONET, SDH, or DWDM, have typically been provisioned 
using a directed approach, where each cross-coimect in the transport path is 
individually established using a vendor-specific element management system 
10 (EMS). As such, end-to-end provisioning of circuits has traditionally been a 
laborious and time-consuming procedure for service providers. 

In a prior art system in Figure 1 , labels have been used to interconnect 
each forwarding engine so as to be distributed throughout the MPLS domain. 

1 5 Traffic associated with these labels would traverse the quasi-static label 
switched paths (LSPs) 140 and 142 used to interconnect devices within the 
access network, so there are no issues related to label proliferation and usage 
of VT space on the SONET ring. In Figure 1, prior art system 100 includes 
routers 120, 122, and 124, servers 130, 132, 134, and 136, an add/drop 

20 multiplexer (MUX) 112, logically interconnected as shown, in a ring 1 1 0. 
Servers 130 and 132 are logically connected via LSP 140, while servers 134 
and 136 are logically connected via LSP 142. Router 120 is a MPLS router. 
TTiere is a physical liok 150. 

In this scenario, nodes in the access network receiving L3 packets from 

25 attached routers would process the IP header or MPLS label and select an 

outgoing label, used to direct the packet toward the appropriate line card at the 
destination node. This label is th^ pushed onto the label stack. Next, the 
label used to forward the packet to the destination node itself is selected and 
pushed onto the stack. 

30 At the destination node, the switch card, which is an MPLS LSR 

(Label Switched Router), will pop the top label off of the stack and examine 
the underlying label to detCTnine which line card the packet is to be switched 
to. In this way, an interoperable method of forwarding trafBc in the access 

3 
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network is achieved. However, the cost is the overhead of having the switch 
card pop the top label ofiF of the label stack. The functionality necessary for 
this capability is essentially identical to that required for supporting access 
network topologies defined by sparsely connected • 
S In other prior art systems, any connections a£forded service guarantees 

were point-to-point (Transparent Lan Service received Best Effort service). 
As such, the admission control decisions were fairly simple. However, in the 
cases described in this document, anypoint-to-anypoint connectivity with 
service guarantees must now be supported. As a result, any port in the system 

10 can be significantly oversubscribed, due to multiple ports directing traffic to it. 
As such, when connections are established, a method of admission control is 
required to ^sure that the level of oversubscription does not reach a point at 
which the associated service guarantees may be compromised. 

Thus, two levels of admission control are required: the lowest level 

1 5 determines whether the connection can be admitted, based on die existing port 
configuration and resources reserved for the currently established connections 
on those ports. This level of admission control can be implemented in the CR- 
LDP/RS VP Traffic Engineering module, local to each network element 
(residing on the present invention's switch card). Additionally, this level of 

20 admission control should embrace a more sophisticated algorithm than simply 
accounting for the summation of CIR or FIR. 

The higher level of admission control is needed to determine whether 
admitting (or deleting) the connection will cause the bandwidth associated 
with the quasi-static node-interconnect LSPs to change. If so, the bandwidth 

25 of the relevant LSP is modified according to the two standards-based methods 
described above. This level of admission control is most easily placed in an 
element that has a global view of the topology (sudi as the centralized EMS, 
in the cunent software architecture). 

30 SUMMARY OF THE INVENTION 

Hie present invention provides a system and method of binding MPLS 
labels to virtually concatenated SONET/SDH transport connections. A 
system and method of binding MPLS labels to nxVT connections is provided 
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and includes (1) classifying packets, and (2) mapping packets with a same 
forward equivalence class to a same nxVT connection. In a further 
embodiment, the mapping includes (a) associating a certain bandwidth of a 
. connection between an external router and an add-drop multiplexer, between a 
5 source node and a destination node, and (b) creating a nxVT conection 
between the source node and the destination node, thereby creating a label 
switch path between the source node and the destination node. 

hi another embodiment, a system and method of binding MPLS labels 
to nxVT connections is also provided and includes mapping packets with the 
10 same MPLS label to the same nxVT connection. 

BRIEF DESCRIPTION OF TBOE DRAWINGS 

Figure 1 illustrates a prior art MPLS labeling system. 
Figure 2 illustrates MPLS labeling in accordance with an exemplary 
embodiment of the present invention. 

15 

DETAILED DESCRIPTION OF THE INVENTION 
Introduction 

The invention described in co-pending and commonly assigned U.S. 
Patent Application No. (Number to be assigned) with Attomey Docket 

20 Number 55369-014 provides a system and method of virtually concatenating 
VTl.Ss and STS-lsovCT SONET and SDH and WDM. The virtual 
concatenation invention allows users to setup connections or pipes with 
configurable bandwidth over either nxSTS-l/nxAU-3/nxAU-4 or 
nxVT1.5/nxTU-l l/nxTU-12 within a nxSTS-l/nxAU-3/nxAU-4 pipe on an 

25 existing SONET/SDH network. This provides a connection or pipe of 

adjustable bandwidth with a granularity of close to 1 .5 Mbps to fit the needs of 
q)plications. The resulting connection can be treated as a TDM like 
connection. 

By replacing "STS-1" with "AU-3" or "AU-4" and "VT" or "VT1.5" 

30 with "TU-1 1" or "TU-12", the virtual concatenation invention applies to 

nxAU-3/nxAU-4 and nxTU-1 l/nxTU-12 for SDH networks. For simplicity, 

these connections are called "nxVT" for both SONET and SDH networks. By 

5 
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replacing "STS-l" with "VT* or "VT1.5", the virtual concatenation invention 
applies to nxSTS-1 and nxAU-3/nxAU-4. 

On top of the virtual concatenation invention, a dynamic bandwidth 
allocation (DBA) protocol, which is described in co-pending and commonly 
5 assigned U.S. Patent Application No. (Number to be assigned) with Attorney 
Docket Number 55369-015, allows for dynamically changing the throughput 
of all nxVT connections, based on the real-time traffic loads of applications 
using the nxVT connections. The DBA protocol allows for the efScient use of 
the SONET/SDH bandwidth through statistical multiplexing. The same 
1 0 dynamic bandwidth allocation protocol appUes to nxSTS-1 and nxAU- 
3/nxAU-4. 

The virtual concatenation invention provides for virtual concatenation, 
which includes creating a logical connection or pipe by combining multiple, n 
(where n is a positive integer), STS-1 or VT connections or pipes, which may 

IS be contiguous or non-contiguous, into a single connection or pipe, nxSTS- 1 or 
nxVT, respectively, in order to siqsport a connection or pipe with a higher 
throughput than the throughput of die original STS-1 or VT pipes. 

By using virtual concatenation, the present invention uses dynamic 
bandwidth allocation protocol used to dynamically manage bandwidth on a 

20 ring (or graph of rings) topology. This technology allows for more efficient 
capacity utilization on a ring (or multiple rings), by rapidly providing ring 
capacity to individual network elements on demand. As such, high capacity 
utilization is achieved for aibitrary traffic loading patterns. 

The present invention provides a system and method of binding MPLS 

25 labels to virtually concatenated SONET/SDH transport connections. 

By leveraging the control plane semantics of MPLS, the present 
invention effects a more seamless management solution by allowing a device 
to participate as a peer in the topology exchange between attached routers at 
Layer-3 and core transport devices at Layer-2. Embracing this philosophy, the 

30 present invention employs MPLS label switched paths (LSP) to establish 
connections between devices in the transport network (See Intemet 
Engineering Task Force. Extensions to RS VP for LSP Tunnels. IETF Draft 
Document September, 1999. http://www.ietf.org^temet-drafts/draft-ietf- 
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mpls-rsvp-lsp-timneWW.txt,) (See Internet Engineering Task Force. 
Constraint Based LSP Setup Using LDP. IETF Draft Document. September, 
1999. http:/Avww.ietf.org/intemet-drafts/<lraft-ietf-mpls-cr-ldp-03.^ and 
provides a gateway functionality betweoi attached IP or MPLS devices and 
S the internal transport netwoik connections canying traffic, whose available 
ring capacity is managed by the DBA protocol. Operating as an MPLS peer in 
the topology exchange at Layer-3 allows die present invention employs 
components of MPLS (See Internet Engineering Task Force. LSP 
Modification Using CR-ix>P. IETF Draft Document September, 1999. 
10 http://www.ietf.org/intCTnet-drafts/draft-ietf-mpls-crlsp-modify-01 .txt.) to 
allow network operators to dynamically create/destroy transport level 
connections and/or modify their associated Service Level Agreements (SLA). 
Label Binding Issues 

When discussing label binding in the context of the present invention, 
15 the present inventions does the following: 

(1) the pre-establishment of label switched paths (LSPs) used to 
interconnect devices within the metropolitan access network^ where these 
devices support the abiUty to dynamically modify the bandwidth associated 
with these LSPs; 

20 (2) the aggregation of connectionless IP flows or connection- 

oriented MPLS LSP traffic arriving at the edge device (fiom attached routers) 
onto paths that are routed through the access network on one or more of the 
above pre-established LSPs; 

(3) dynamic modification of bandwidth associated with both 
25 the pre-established LSPs and the Service Level Agreements (SLA) associated 

with the connections ftom attached Layer-3 devices; and 

(4) bmding of labels used for LSPs that provide connectivity to 
core transport devices being used to interconnect various metro access 
networks. 

30 Referrring to Figure 2, by embracmg the work underway in the IETF, 

edge devices in transport level access networks can exist as peers in the 
topology exchange of core transport devices via an interoperable, standards- 
based method. Therefore, the label bindings used to resolve the last buUet 

7 
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item in the list above is considered beyond the scope of this document. What 
is of particular relevance to the content of this document is the resolution of 
the issues described in the first two bullet items. For the purpose of this 
discussion, consider tiiie network topology described by N network elements, L 
S line cards per N, P interfaces (physical or logical) per L, and F queueing 
subclasses (denoting an MPLS Forward Equivalence Class, or FEC) per P. 
LSPs Used to Interconnect NEs 

In the present invention, the worst-case number of LSP required to 
interconnect network elemrats on each ring is calculated according to the 

10 well-known formula ((( N * ( N- 1 )) / 2 ). Note that the graph can contain 
fewer vertices, in order to save VT space on the ring. However, in the case 
where a connected graph is not established, the network operator must 
establi^ an explicitly routed LSP tunnel to route traffic between endpouits of 
a connection that do not share an adjacency. Additionally, in the case where 

15 the graph is not connected, packets traversing these LSP tunnels will endiure 
increased delay and delay variance as a result of the increased hop count 
Perhaps more importantly, this case requires that the present invention push 
and pop labels on the label stack, which will require CPU intervention. 
Because the present invention's line and switch cards provide QoS-capable 

20 forwarding of packet traffic, no dilation of the basic node interconnect graph is 
required. Interconnecting networic elements to those of other vendors may 
require the use of additional LSPs; one per FEC, for example. 



8 
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These LSPs can be configured using either RSVP-LDP or CR-LDP. 
The essential behavior of these LSPs should be similar in nature to those of 
ATM Soft-PVCs. For the general case, the bandwidth allocated for each LSP 
is calculated according to the following formula: 

5 

LP LP 
BWLSP = inax(( S E BW,j >rxEndpointW ( 2 Z BWy h-xEndRoina) 



1 0 For the case where only symmetric connections (where the specified 

bandwidth must be idmtical in each direction) are supported, the above 
^ression is constrained such that each endpoint must have equivalent values. 
This constraint alleviates the necessity for any proprietary messaging to pass 
between the endpomts of an LSP, used to signal each endpoint's 

15 corresponding component ofthe above expression. Thus, for the 

deconstructed case, where only symmetric virtually concatenated connections 
are supported, the above expression reduces to: 

L P 

BWlsp = ( li 2 BWij ) 

20 

In the simple case, the BW componrat of the summation above is 
simply the PIR associated with each coimection on each line card of a network 
element. The accuracy of the vahie B W could be improved by enqiloying a 
25 slightly more sophisticated approach, such as the well-known equivalent 

bandwidth calculation [Guerin91], which accounts for the effects of statistical 
multiplexing and aggregate utilization, as well as employing a more realistic 
source model However, because DBA is managing the bandwidth usage 
within these LSPs, the amount of oversubscribing actually occurring is not of 
30 particular concern. 

Dypamic Modi^cation ^sy Ryodification 
It should be noted that, when connections are added or removed firom 
the network, such that the bandwidth derived firom the above expressions 
changes, the bandwidth associated with the LSP is dynamically adjusted. 
35 LSP ModincatioD Using CR-LDP 
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Modification of the bandwidth associated with an existing LSP begins 
with the issuance of a LDP Label Request Message by the node requiring 
more bandwidth. Within this message, the encoded Type-Length- Value 
(TLV) will contain the newly requested bandwidth values^ and the LSP 
5 identifier TLV (LSPID TLV). On transmission of the Label Request Message, 
the sending node will retain the original entry in its FEC to Next Hop Label 
Forwarding Entry (FTN) table. 

On receipt of the Label Request Message, the receiving node will 
detect that the LSPID is identical to that of one of tiie ratries in its Incoming 
10 Label Map (ILM). If the receiving node did not find such an entry in its ILM, 
it would simply treat the Label Request Message as a new label request, and 
process the request in the typical fashion. However, in this case, it will 
compute the new bandwidth required. The receiving node will reserve only 
the differrace between the original and new bandwidth values, in order to 
1 5 prevent double booking the capacity. As such, this node will temporarily have 
two labels in its ILM with the same LSPID. 

Similarly, when the initiating node receives the label from its LDP 
peer, two sets of labels will exist for the same LSPID, As such, the initiating 
node will now have two outgoing labels for the same FEC. The initiating 
20 node can then activate the new label in its FTN and begin the process of 

releasing the old label, by sending an LDP Label Release Message to its peer. 

On receipt of the Label .Release Message, the egress node will detect 
that the LSPID has anothCT active label associated with it. As a result, this 
node will release the old label without deallocating the resources being used 
25 by that label. 

LSP Modification Using RSVP^LDP 

The method of LSP bandwidth modification using RSVP-LDP is 
virtually identical to that used by CR-LDP, as this portion of CR-LDP is based 
on previous work in RSVP. However, the specific details of this fimctionality 
30 in RSVP are left for fiiture study. 

Anvpoint-to-Anvpoint Laver-3 Connectivity 
Iliis type of connectivity is provided through the use of Layer-3 
routing and forwarding. As the L3 forwarding functionality resides on line 

10 
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cards in the present invention, each packet must contain information that the 
switch card can use to determine the appropriate line card at the destination 
endpoint 

The present invention appends a tag to the packet by the Layer-3 
5 forwarding engine to specify the line card. This is essentially a tag switching 
approach. This method will work well on homogeneous rings employing only 
devices of the present invention. In this approach, the number of tags needed 
is calculated as the product of tfie total number of NEs (N - 1), line cards (L), 
andFECs(F): 



10 



NUMtags = Fx((N-1 )xL) 



For the current design, this implies ( 4 x ( 31 x 8 )) = 992 proprietary 
IS tags (10 bits). 

Dynamic Resizing of Virtnallv Concatenated Connections 
One of the essratial benefits of the present invention functioning as an 
MPLS LSR lies in its ability to leverage CR-LDP or RSVP-LDP for 
dynamically resizing the dimension N of NxVT, NxTUl 1, or NxSTS-1 
20 cormections. This cs^abiUty provides a mechanism by which attached LSRs 
residing at the service provider's Point-of-Presence (POP) can signal the 
creation, destruction, or resizing of coimections on an end-to-end (POP-to* 
POP) basis. This capability is afforded by the ability of the present invention 
to fimction as an LDP peer with attached LSRs. 
25 Note ttiat, in the case where the attached router is not an LSR, the 

method available for dynamic resizing of virtually concatenated cormections 
will leverage native RSVP over IP, rather than CR-LDP or RSVP-LDP. 



Conclusion 

30 The present invention relates to optical networking. More particularly, 

the invention relates to a system and method of binding MPLS labels to 
virtually concatenated SONET/SDH transport coimections. 

Having fiilly described a preferred embodiment of the invration and 
various alternatives, those skilled in the art will recognize, given the teachings 

11 
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herein, that numerous alternatives and equivalents exist which do not depart 
from the invention. It is therefore intended that the invention not be limited by 
the foregoing description, but only by the appended claims. 



12 
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CLAIMS 

We claim: 

1 . A method of binding MPLS labels to nxVT comiections 
comprising: 

5 classiiying packets; and 

mapping packets with a same forward equivalence class to a 
same nxVT coxmection. 

2. The method of claim 1 wherein the classifying comprises 
1 0 identifying a packet by a packet field of the packet. 

3. The method of claim 2 wherein the identifying comprises, if the 
packet is a Internet Protocol packet, examining the DHCP byte of the packet. 

IS 4. The method of claim 2 wherein the identifying comprises, if the 

packet is an PPP packet, examining PPP information of the packet. 

5. The method of claim 2 wherein (be identifying comprises, if the 
packet is an ATM packet, examining the VPI or VCI of the packet. 

20 

6. The method of claim 2 wherein the identifying comprises, if the 
packet is a Frame Relay packet, examining the DLQ of the packet 

7. Hie metfiod of claim 1 wherein the mapping comprises: 

13 
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associating a certain bandwidth of a connection between an 
external router and an add-drop multiplexer, between a source node and a 
destination node; and 

creating a nxVT conection between the source node and the 
5 destination node, thereby creating a label switch path between the source node 
and the destination node. 



8. A method of binding MPLS labels to nxVT connections 
comprising mapping packets with the same MPLS label to the same nxVT 
10 connection. 



9. The method of claim 8 wh^ein the mapping comprises: 

associating a c^ain bandwidth of a connection between an 
external router and an add-drop multiplexer, between a source node and a 
1 5 destination node; and 

creating a nxVT conection between the source node and the 
destination node, thereby creating a label switch path between the source node 
and the destination node. 



20 
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